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Dear  Mr.  Chairman: 


For  more  than  30  years,  the  Federal  Aviation  Administration  (faa)  and  the 
aviation  industry  have  been  working  to  develop  a  system  to  help  prevent 
mid-air  and  near  mid-air  collisions.  In  1981,  after  evaluating  several 
systems,  faa  decided  to  develop  and  deploy  the  Traffic  Alert/Collision 
Avoidance  System  (tcas).1  tcas  is  an  airborne,  aircraft-to-aircraft  system 
that  scans  surrounding  airspace,  warns  of  potential  intruders,  and 
recommends  evasive  maneuvers. 

As  you  requested,  this  report  discusses  (1)  pilots’  and  air  traffic 
controllers’  views  on  tcas,  (2)  faa’s  actions  to  address  tcas’s  problems, 
and  (3)  key  aspects  of  faa’s  software  engineering  approach  for  tcas, 
including  faa’s  plans  to  verify  and  validate  the  system.2 


Results  in  Brief 


tcas  is  now  installed  in  a  substantial  portion  of  the  U.S.  commercial  fleet, 
and  both  the  Airline  Pilots  Association  and  faa  believe  that  the  system 
adds  a  margin  of  safety  to  air  travel.  However,  problems  that  have 
emerged  prevent  the  system  from  reaching  its  full  potential.  The  aviation 
community  is  nearly  unanimous  in  recognizing  that  tcas  needs  to  be 
improved  because  it  issues  too  many  unnecessary  alerts,  causes  excessive 
altitude  deviations  (over  1,000  feet),  and  causes  pilots  to  miss  landing 
approaches.  Pilots  and  air  traffic  controllers  stated  that  these  problems 
reduce  users’  confidence  in  tcas  and  the  margin  of  safety  that  the  system 
provides. 


‘FAA  expects  to  have  three  TCAS  models.  TCAS  I,  the  least  costly  and  least  technically  sophisticated, 
recommends  no  collision  avoidance  maneuvers  and  is  being  designed  for  small  commercial  and 
general  aviation  aircraft  TCAS  II  and  III  are  intended  primarily  for  larger  commercial  air  carriers. 
TCAS  II  recommends  vertical  avoidance  maneuvers;  TCAS  III  is  under  development  and  is  expected  to 
recommend  both  vertical  and  horizontal  maneuvers.  TCAS  II  is  the  subject  of  this  report 

Verification  ensures  that  a  product  conforms  to  specified  requirements,  while  validation  ensures  that 
the  product  completely  and  correctly  meets  users’  needs. 
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faa  plans  to  reduce  unnecessary  alerts  by  modifying  tcas  and  will 
complete  a  series  of  interim  computer  simulation  tests  and  a  safety  study 
to  ensure  that  the  modifications  can  be  introduced  safely  in  the  system. 
faa  plans  to  make  the  modifications  available  at  the  end  of  March  1992. 
However,  because  faa  omitted  some  steps  in  verifying  and  validating  tcas 
before  authorizing  its  installation  in  commercial  aircraft,  faa  still  has  to 
complete  this  process  and  plans  to  do  so  by  the  end  of  1992. 

Because  faa’s  planned  modifications  would  delay  tcas  alerts  until 
intruding  aircraft  are  closer,  some  members  of  the  Separation  Assurance 
Task  Force — a  tcas  review  committee  comprising  representatives  of 
pilots,  controllers,  and  avionics  and  airframe  manufacturers — believe  that 
faa  should  fully  verify  and  validate  tcas  and  the  modifications  before 
implementing  the  modifications.  However,  other  task  force  members  and 
faa  believe  that  neglecting  the  current  problems  reduces  pilots’  confidence 
and  presents  a  greater  risk;  therefore,  according  to  these  members,  the 
modifications  should  be  implemented  immediately.  We  see  no  clear-cut 
answer  to  this  dilemma — both  points  of  view  have  merits  and  entail  risks. 

Since  faa  plans  to  implement  the  modifications  before  completing 
verification  and  validation,  we  believe  that,  at  a  minimum,  users  should 
have  an  opportunity  to  review  the  modifications’  interim  test  methodology 
and  results.  During  the  latter  stages  of  our  review,  faa  decided  to  give  the 
Separation  Assurance  Task  Force  an  opportunity  to  review  and  comment 
on  the  test  methodology  and  results  before  modifying  the  system.  Such  a 
step  is  critical  to  ensure  that  users’  problems  are  identified  and  corrected 
and  to  bolster  users’  confidence  in  the  safety  of  the  modifications,  even  if 
addressing  problems  causes  the  tcas  installation  schedule  to  slip. 


Background 


The  Airport  and  Airway  Safety  and  Capacity  Expansion  Act  of  1987 
required  that  all  commercial  aircraft  with  over  30  passenger  seats  be 
equipped  with  tcas  by  December  30, 1991.  A  subsequent  amendment  gave 
the  faa  Administrator  the  discretion  to  extend  the  deadline  to  December 

30. 1993.  Accordingly,  faa  called  for  installing  tcas  in  20  percent  of  the 
designated  aircraft  by  December  30, 1990;  50  percent  by  December  30, 
1991;  and  the  remainder  by  December  30, 1993.  As  of  December  30, 1991, 
48  percent  of  the  designated  aircraft  were  equipped  with  tcas.  Airlines  are 
continuing  to  install  tcas  to  achieve  100-percent  installation  by  December 

30. 1993,  according  to  faa’s  tcas  Program  Manager. 
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tcas  backs  up  pilots’  vision  and  air  traffic  controllers’  monitoring  systems 
to  ensure  safe  separation  between  aircraft,  tcas’s  cockpit  display  shows 
surrounding  air  traffic,  and  the  system  verbally  warns  flight  crews  of 
potential  and  actual  intrusions.  Three  companies — Bendix/King; 
Honeywell,  Inc.;  and  Rockwell  Collins — currently  manufacture  and  market 
tcas.  Figure  1  shows  a  typical  tcas  display  and  instrumentation. 


Using  aircraft  transponder  signals,3  tcas  estimates  whether  one  or  more 
aircraft  are  likely  to  enter  an  aircraft’s  protected  space.  Initially,  tcas 
issues  a  verbal  “Traffic,  Traffic”  warning — called  a  traffic  advisory — if 


3A  transponder  is  a  device  that  receives  and  transmits  a  radio  signal  for  air  traffic  control  purposes. 
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another  aircraft  comes  within  1,200  feet.  Flight  crews  use  this  warning  and 
tcas’s  cockpit  display  to  see  the  intruder.  If  the  intruder  continues  to 
converge,  tcas  will  issue  a  resolution  advisory  when  the  aircraft  is  within 
400  feet  at  altitudes  under  10,000  feet,  500  feet  at  altitudes  between  10,001 
and  20,000  feet,  640  feet  at  altitudes  between  20,001  and  30,000  feet,  and 
740  feet  at  altitudes  over  30,000  feet.  The  resolution  advisory  verbally  tells 
the  pilot  to  climb  or  descend  and  displays  the  recommended  rate  of  climb 
or  descent  on  cockpit  instrumentation.  The  resolution  advisory  suggests  a 
maneuver  that  is  designed  to  maintain  safe  separation  between 
transponder-equipped  aircraft  on  the  basis  of  tcas’s  calculations  of  their 
range,  altitude,  and  relative  speed,  tcas  can  be  operated  to  provide  traffic 
advisories  only,  or  both  traffic  and  resolution  advisories.  When  providing 
traffic  advisories  only,  tcas  will  not  advise  the  pilot  to  climb  or  descend. 

faa  and  industry  software  engineering  standards  call  for  rigorous 
verification  and  validation  throughout  a  system’s  development  and  testing. 
(App.  II  lists  these  standards.)  Verification  and  validation  are  common 
procedures  for  minimizing  risks  on  critical  systems  whose  use  could  result 
in  loss  of  life  or  some  other  catastrophic  event.  Verification  and  validation 
involve  analyzing  and  testing  software  throughout  its  life  cycle  to  ensure 
performance,  integrity,  reliability,  safety,  and  quality.  Although  the 
distinction  between  verification  and  validation  is  sometimes  blurred,  we 
use  verification  to  mean  the  steps  taken  to  ensure  that  a  product  meets 
specified  requirements.  We  use  validation  to  mean  the  steps  taken  to 
ensure  that  the  product  completely  and  correctly  meets  users’  needs. 


Aviation  Industry  Has 
Mixed  Opinions  on 
TCAS’s  Benefits 


The  Airline  Pilots  Association  and  faa  officials  agree  that  tcas  has 
increased  the  margin  of  safety  in  aviation  travel.  An  Airline  Pilots 
Association  representative  told  us  that  the  more  experience  pilots  have 
with  tcas,  the  more  they  like  it.  The  representative  also  said  that  pilots 
would  revolt  if  tcas  were  removed  from  the  cockpit,  faa  cited  instances  in 
which  tcas  has  helped  to  resolve  or  prevent  potentially  serious  situations. 
In  one  instance,  an  aircraft  equipped  with  tcas  was  approaching  a  busy 
airport  when  it  received  a  traffic  advisory.  When  the  traffic  advisory 
changed  to  a  resolution  advisory,  the  crew  began  to  climb.  When  clear  of 
the  potential  danger,  the  crew  leveled  off  and  saw  the  intruding  aircraft 
pass  500  feet  below.  This  aircraft  had  not  appeared  on  the  controller’s 
radar  screen.  In  another  instance,  tcas  helped  avoid  a  potential  collision 
between  a  Boeing-747  and  a  DC-10  traveling  in  opposite  directions  in 
darkness  over  the  Pacific  Ocean. 
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The  individual  pilots  and  controllers  we  interviewed  gave  mixed  reviews 
of  tcas.  A  number  of  pilots  said  they  believed  that  tcas  is  a  good  system  or 
told  us  of  incidents  in  which  tcas  had  helped  them  avoid  other  aircraft. 
Some  pilots,  however,  were  less  complimentary,  stating  that  tcas 
interrupts  normal  flight  procedures  in  the  cockpit.  Most  of  the  38 
controllers  whom  we  interviewed  agreed  that  tcas  is  a  good  concept  but 
stated  that  problems  exist.  The  National  Air  Traffic  Controllers 
Association  claims  that  because  tcas  disrupts  air  traffic,  the  system  has 
thus  far  had  a  negative  effect  on  air  safety.  However,  the  association 
believes  that  as  existing  problems  are  corrected,  tcas  could  enhance  air 
safety. 


Pilots  and  Controllers 
Report  Problems  With 


TCAS 


To  evaluate  tcas’s  operational  effectiveness  and  identify  problems,  faa 
asked  pilots  and  air  traffic  controllers  to  complete  questionnaires  about 
their  experience  with  the  system.  Between  June  1990  and  October  1991, 
faa  received  responses  from  about  2,400  pilots  and  1,700  controllers.  The 
responses  identified  three  irayor  concerns:  (1)  tcas  has  issued  some 
resolution  advisories  that  have  caused  pilots  to  unnecessarily  leave 
assigned  aiiport  approaches,  go  around  airports,  and  reenter  landing 
patterns  (30  instances),  (2)  pilots  have  made  large  altitude  deviations 
(over  1,000  feet)  in  response  to  tcas  (86  instances),  and  (3)  tcas  has  issued 
unnecessary  advisories  while  pilots  were  following  established  air  traffic 
control  procedures  (359  instances). 


Controllers’  responses  indicate  a  major  concern  about  tcas’s  impact  on 
their  operations.  Controllers  stated  that  the  altitude  deviations  can  cause 
an  aircraft  to  deviate  into  another  sector,  requiring  rapid  coordination 
between  controllers,  which  increases  their  workload — a  situation  that 
controllers  believe  is  unacceptable  when  air  traffic  is  heavy.  Controllers 
also  claimed  that  the  alerts  increase  communication  between  pilots  and 
controllers  when  pilots  ask  about  the  alerts,  which  places  additional  stress 
on  controllers,  especially  during  high  traffic  periods.  A  National  Air  Traffic 
Controllers  Association  representative  stated  that  the  unnecessary  alerts 
diminish  pilots’  and  controllers’  confidence  in  the  system. 


The  National  Air  Traffic  Controllers  Association  and  faa’s  Associate 
Administrator  for  Air  Traffic  also  stated  that  excessive  altitude  deviations 
and  pilots’  responses  to  unnecessary  alerts  disrupt  traffic  and  contribute 
to  delays.  The  Association  believes  that  tcas  was  deployed  too  early,  and 
most  controllers  whom  we  interviewed  do  not  believe  that  faa  tested  tcas 
sufficiently  to  determine  its  impact  on  air  traffic  control  operations.  Both 
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groups  believe  that  pilots  should  use  only  the  traffic  advisory  feature  until 
the  problems  are  resolved.  Such  action,  the  Association  says,  would 
eliminate  the  resolution  advisories  that  cause  pilots  to  move  out  of 
assigned  flight  paths. 

An  faa  contractor’s  analysis  of  the  questionnaire  responses  pointed  out 
some  factors  that  contribute  to  the  excessive  altitude  deviations.  Some 
pilots  mistakenly  believe  that  a  “climb”  or  “descend”  resolution  advisory 
requires  movement  to  the  next  legal  altitude  level,  which  is  1,000  feet 
above  or  below  their  assigned  level.  Pilots  may  also  not  recognize  or 
respond  appropriately  to  tcas’s  indication  that  they  are  clear  of  another 
aircraft  and  may  therefore  continue  their  climb  or  descent  unnecessarily. 
These  issues  point  to  training  concerns  that  faa  and  airline  publications 
have  repeatedly  addressed.  In  addition,  some  reported  deviations  were 
found  to  be  exaggerated,  faa’s  contractor  analyzed  23  reported  altitude 
deviations  in  excess  of  1,000  feet  and  found  that  only  5  were  actually  over 
1,000  feet. 

Pilots’  responses  typically  addressed  tcas’s  annoying  repeated  verbal 
alerts  against  the  same  known  threat  and  indicated  that  the  unnecessary 
alerts  reduce  their  confidence  in  the  system,  faa  also  believes  that  these 
problems  are  causing  some  pilots  to  lose  confidence  in  tcas  and  to  either 
ignore  or  turn  off  the  system  during  final  approach.  Such  actions,  faa  says, 
eliminate  the  margin  of  safety  that  tcas  provides.  The  problems  that  pilots 
cited  in  the  questionnaire  responses  were  also  mentioned  by  many  of  the 
pilots  whom  we  interviewed.  Some  also  said  that  they  turn  the  system  off 
to  avoid  unnecessary  alerts. 

Two  reasons  have  been  cited  for  the  unnecessary  advisories.  First,  tcas 
can  sound  a  traffic  advisory  when  vertical  separation  between  aircraft  is 
projected  to  be  less  than  1,200  feet,  whereas  standard  air  traffic  control 
procedures  allow  separations  of  as  little  as  500  feet.  Second,  air  traffic 
control  at  some  airports  directs  aircraft  to  take  off  and  climb  to  altitudes 
below  other  aircraft.  In  such  instances,  the  aircraft  remain  separated  in 
accordance  with  normal  standards.  However,  because  tcas  computes 
potential  conflicts  on  the  basis  of  an  intruding  aircraft’s  trajectory  and  rate 
of  speed,  the  system  cannot  anticipate  that  the  climbing  aircraft  will  level 
off.  Therefore,  tcas  issues  a  resolution  advisory. 
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Validation  involves  users’  reviewing  a  system’s  initial  specifications  to 
ensure  that  the  system  will  meet  users’  needs  when  it  is  built.  The 
validation  process  includes  a  formal  review  of  a  system’s  initial 
specifications  and  a  formal,  documented  resolution  of  users’  concerns  to 
ensure  that  the  system  will  meet  users’  needs.  Verification  involves  testing 
the  system  to  ensure  that  it  performs  according  to  its  specifications. 

Software  engineering  standards  in  place  when  tcas’s  specifications  were 
being  developed  recognized  the  importance  of  verification  and  validation 
in  developing  critical  software-intensive  systems.4  Because  verification  and 
validation  occur  throughout  development,  they  can  identify  problems  at 
any  time.  However,  these  processes  are  designed  to  identify  problems 
early,  when  they  are  easier  and  less  costly  to  fix. 

Before  installing  tcas  in  commercial  aircraft,  faa  verified  that  tcas 
performed  in  accordance  with  its  specifications  by  conducting  thousands 
of  simulated  air  traffic  scenarios  and  about  6,200  hours  of  flight  tests  from 
1982  to  1989.  These  tests  confirmed  that  tcas  performed  according  to  its 
specifications.  However,  faa  did  not  validate  a  key  element  of  tcas — the 
collision  avoidance  requirements.  According  to  a  tcas  manufacturer,  no 
system-level  specifications  were  developed  for  this  portion  of  the  system. 
System-level  specifications  facilitate  validation  because  they  are  written  in 
terms  that  users  can  easily  understand. 

Rather  than  developing  system-level  specifications,  faa  and  the  aviation 
industry  defined  the  collision  avoidance  requirements  in  pseudocode,  a 
detailed  specification  language  similar  to  a  high-level  programming 
language,  faa  believed  that  pseudocode  was  necessary  to  help  ensure  that 
each  of  the  three  tcas  manufacturers’  systems  coordinated  collision 
avoidance  maneuvers  correctly  with  the  other  manufacturers’  systems. 
Pseudocode  helped  ensure  coordination  among  three  manufacturers’ 
versions  because  it  defined  the  collision  avoidance  specifications  precisely 
and  unambiguously,  thereby  providing  the  manufacturers  with  little 
leeway  in  interpreting  the  specifications.  This  precision,  in  turn,  ensured 
that  the  manufacturers  would  interpret  the  specifications  correctly  and 
consistently. 

Pseudocode  also  allowed  faa  to  verify  that  tcas  met  its  specifications. 
From  this  perspective,  faa’s  software  engineering  approach  was  judicious 


‘Software  Considerations  in  Airborne  Systems  and  Equipment  Certification,  Radio  Technical 
Commission  for  Aeronautics  (KTCA/DO-178,  Nov.  1981)  and  Guideline  for  Lifecycle  Validation, 
Verification,  and  Testing  of  Computer  Software,  Federal  Information  Processing  Standard  101  (June 
15S3}: 


An  Important  Element 
of  TCAS’s  Software 
Was  Not  Fully 
Validated 
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and  effective.  However,  the  absence  of  system-level  specifications  for 
tcas’s  collision  avoidance  requirements  limited  the  opportunities  for  users 
to  participate  in  tests  for  ensuring  that  the  system  would  meet  their  needs. 
faa  has  since  recognized  the  need  to  develop  system-level  specifications 
and  perform  full  verification  and  validation  of  tcas  and  is  taking  steps  to 
complete  both  by  the  end  of  1992. 


Unnecessary  Alerts 
Were  Identified  Late 
in  the  Process  and  Not 
Effectively  Resolved 


Although  validating  tcas’s  collision  avoidance  requirements  might  have 
identified  the  unnecessary  alerts  early  in  tcas’s  development,  the  problem 
emerged  later  as  faa  verified  tcas  during  flight  testing.  In  this  regard,  faa’s 
testing  program  was  effective.  However,  faa  did  not  adequately  resolve  the 
problem  once  it  surfaced.  Specifically,  after  users  pointed  out  tcas’s 
unnecessary  alerts,  faa  and  the  aviation  industry  chose  not  to  conduct 
further  testing  to  evaluate  the  significance  of  the  problem  because  of  a 
congressional  mandate  to  install  tcas  in  the  entire  aircraft  fleet  by 
December  30, 1991.  At  that  time,  legislation  extending  the  installation 
deadline  to  December  30, 1993,  had  not  been  passed,  faa’s  tcas  Program 
Manager  believes  that  although  the  test  results  adequately  ensured  tcas’s 
safety,  further  testing  might  have  allowed  faa  to  better  assess  the 
significance  of  the  problem.  In  view  of  the  legislative  deadline,  faa  and 
industry  representatives  decided  to  implement  a  procedural 
“work-around”  that  gave  pilots  the  option  to  operate  tcas  in  the  traffic 
advisory  mode  only  during  final  approach  and  in  certain  other 
circumstances. 


This  action  did  not  solve  the  problem  because  some  airlines  required 
pilots  to  operate  tcas  in  the  resolution  advisory  mode  throughout  flight.  In 
addition,  when  in  the  traffic  advisory  mode,  tcas  still  issues  the  repeated 
verbal  alerts  against  other  aircraft  while  routine  separation  is  being 
maintained.  Hence,  the  unnecessary  alerts  and  deviations  continue  to  be  a 
mEyor  source  of  discontent  among  pilots  and  controllers.  To  address  these 
problems,  faa  will  modify  tcas’s  specifications  by  the  end  of  March  1992. 


FAA’s  Plan  to  Modify 
TCAS  Generates 
Controversy 


Opinion  within  the  aviation  industry  is  sharply  divided  on  whether  faa 
should  introduce  the  modifications  to  tcas  before  completing  full 
verification  and  validation,  faa  plans  to  provide  modified  specifications  to 
tcas  manufacturers  in  March  1992  but  will  not  complete  verification  and 
validation  until  the  end  of  1992.  faa  expects  the  modifications  to  reduce 
the  length  of  time  and  distance  at  which  tcas  sounds  alerts,  bringing  tcas’s 
parameters  more  in  line  with  air  traffic  control  separation  standards. 
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Below  altitudes  of  5,000  feet,  warning  time  for  pilots  would  be  reduced  by 
20  to  25  percent.  For  example,  at  an  altitude  of  1,500  feet,  tcas  currently 
provides  a  resolution  advisory  20  seconds  before  an  aircraft  is  projected  to 
approach  within  400  feet,  whereas  under  the  modifications  the  warning 
would  occur  only  15  seconds  before  the  projected  approach.  According  to 
faa  and  tcas  manufacturers,  these  modifications  do  not  require  significant 
software  changes  and  no  new  functions  are  being  introduced. 


FAA  and  Some  Industry 
Members  Believe 
Modifications  Should  Not 
Be  Delayed 


faa  officials  stated  that  ongoing  computer  simulations  and  a  safety  study, 
which  will  be  completed  before  the  modifications  are  made  available  to 
tcas  manufacturers,  will  demonstrate  that  the  modifications  will  not 
adversely  affect  safety  and  that  modified  and  unmodified  systems  will 
interact  properly.  Therefore,  faa  believes  that  the  changes  can  be 
introduced  safely  before  tcas  has  been  fully  verified  and  validated. 

One  of  the  three  tcas  manufacturers  plans  to  proceed  with  the 
modifications  in  the  belief  that  faa’s  testing  will  have  been  adequate  to 
demonstrate  their  safety.  The  National  Air  Traffic  Controllers  Association 
also  sides  with  faa  because  it  wants  to  see  an  end  to  unnecessary  traffic 
disruptions  caused  by  tcas  alerts. 


faa  officials  believe  that  the  modifications  should  not  be  delayed  until  faa 
has  fully  verified  and  validated  tcas  (i.e.,  completed  all  verification  and 
validation  steps  previously  omitted).  They  believe  that  deferring  the 
modifications  would  further  erode  pilots’  confidence  in  tcas  and  increase 
the  number  of  pilots  ignoring  or  turning  off  the  system,  faa’s  tcas  Program 
Manager  believes  that  a  similar  loss  of  confidence  by  pilots  in  another 
avionics  system,  the  ground  proximity  warning  system,  resulted  in  a  1978 
crash  at  Pensacola,  Florida.  According  to  a  National  Transportation  Safety 
Board  report,  the  flight  engineer’s  turning  off  the  ground  proximity 
warning  system  contributed  to  the  accident. 


Some  Industry  Members 
Believe  Modifications 
Should  Be  Delayed 


Some  members  of  the  Separation  Assurance  Task  Force — a  tcas  review 
committee  comprising  representatives  of  pilots,  controllers,  tcas 
engineers  and  manufacturers,  airlines,  and  a  nuyor  airframe 
manufacturer — believe  that  reducing  aircraft  separation  and  warning  time 
for  pilots  will  create  an  unacceptable  risk.  They  believe  that  implementing 
the  modified  tcas  should  be  deferred  until  tcas  has  been  fully  verified  and 
validated  in  accordance  with  widely  accepted  software  engineering 
practices.  They  also  noted  that  the  planned  modifications  involve  changes 
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to  the  software  that  may  introduce  new  problems  in  other  parts  of  the 
system.  They  said  that  until  tcas  has  been  fully  verified  and  validated,  they 
could  tolerate  the  existing  distractions  rather  than  expose  themselves  to 
new  risks. 

Other  members  of  the  aviation  industry  expressed  similar  views. 
Representatives  of  the  Boeing  Commercial  Airplane  Group  told  us  that 
they  are  opposed  to  modifying  tcas  before  it  has  been  fully  verified  and 
validated.  Representatives  from  two  tcas  manufacturers  said  that  they  do 
not  plan  to  adopt  the  modifications  because  the  airlines  they  service  do 
not  believe  that  the  false  alerts  are  serious  enough  to  warrant  the 
increased  risks  inherent  in  reduced  aircraft  separation  and  reduced 
warning  time  for  pilots.  Because  these  two  manufacturers  do  not  plan  to 
incorporate  the  modifications,  some  Separation  Assurance  Task  Force 
members  expressed  concern  that  collision  avoidance  coordination 
problems  may  arise  between  modified  and  unmodified  systems. 

During  the  latter  stages  of  our  review,  faa  decided  to  give  the  task  force  an 
opportunity  to  comment  on  and  question  the  test  methodology  and  results 
before  providing  the  modifications  to  tcas  manufacturers.  If  the  process 
raises  significant  concerns,  the  modifications  will  be  delayed  until  the 
problems  have  been  resolved,  faa  believes  that  this  step  is  necessary  to 
enhance  users’  acceptance  of  the  modifications,  faa  had  initially  planned 
to  brief  the  task  force  after  providing  the  modifications  to  tcas 
manufacturers. 


Conclusions 


Because  tcas  directly  affects  air  safety,  those  associated  with  the  system 
hold  strong  opinions  concerning  its  benefits.  Most  members  of  the  aviation 
industry  share  the  opinion  that  the  concept  is  good  but  differ  in  their  views 
on  the  significance  of  the  problems  reported  in  implementing  tcas. 
Operational  experience  to  date  has  shown  that  human  factors — pilots’ 
confidence  in  tcas  and  controllers’  acceptance  of  the  system — are 
important  to  its  success.  We  believe  that  faa  should  have  fully  verified  and 
validated  tcas  before  authorizing  its  installation  in  commercial  aircraft, 
and  we  endorse  faa’s  plans  to  complete  the  full  verification  and  validation 
process  at  the  earliest  possible  date. 

A  number  of  tcas  users  are  skeptical  that  the  planned  modifications  can  be 
introduced  safely  before  full  verification  and  validation  have  been 
completed,  faa’s  plans  to  allow  the  Separation  Assurance  Task  Force  to 
comment  on  the  testing  methodology  and  results  before  faa  provides  the 
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modifications  to  tcas  manufacturers  should  help  alleviate  this  skepticism 
and  strengthen  users’  confidence  in  the  safety  and  effectiveness  of  the 
modifications.  Although  responding  to  issues  that  may  arise  during  this 
review  could  delay  faa’s  scheduled  release  of  the  modifications  in  March 
1992,  we  believe  that  adequate  attention  to  users’  concerns  is  essential  to 
gaining  the  aviation  community’s  acceptance  of  the  modifications. 

As  tcas  is  installed  in  the  remainder  of  the  fleet,  other  system  difficulties 
affecting  safety  could  emerge,  just  as  the  unnecessary  alerts  occurred 
during  testing,  faa  tried  to  address  that  problem  in  a  manner  that  allowed 
it  to  meet  a  legislative  deadline  for  installing  tcas.  Because  the  solution 
was  not  entirely  effective,  faa  is  now  faced  with  modifying  systems 
already  installed  in  approximately  half  the  commercial  fleet.  If  similar 
operational  difficulties  occur  in  the  future,  faa’s  recent  experience  may 
suggest  the  advisability  of  delaying  further  installation  or  possibly 
rendering  existing  units  inoperative  until  the  problems  have  been  resolved. 
Although  such  actions  could  require  faa  to  seek  an  extension  of  the 
December  30, 1993,  deadline  for  installing  tcas  in  all  aircraft,  such  a 
trade-off  in  the  name  of  safety  would  appear  prudent. 


We  recommend  that  the  Secretary  of  Transportation  direct  the  faa 
Administrator  to  follow  through  on  current  plans  to  (1)  fully  verify  and 
validate  all  future  significant  modifications  of  tcas,  (2)  effectively  involve 
tcas  users  and  other  interested  parties  in  testing  modifications  through 
commenting  on  and  questioning  the  test  methodology  and  results,  and  (3) 
address  all  users’  concerns. 


We  discussed  the  facts  presented  in  this  report  with  faa  officials,  who 
generally  agreed  with  the  facts  but  disagreed  with  our  statements 
concerning  users’  lack  of  involvement  in  tcas’s  initial  design.  In  response 
to  faa’s  concerns,  we  gathered  additional  data  and  incorporated  the 
results.  As  requested,  we  did  not  obtain  written  agency  comments  on  a 
draft  of  this  report. 


We  conducted  our  review  from  July  1991  to  January  1992  in  accordance 
with  generally  accepted  government  auditing  standards.  A  detailed 
discussion  of  our  objectives,  scope,  and  methodology  appears  in 
aippendix  I. 


Recommendation 


Agency  Comments 
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Unless  you  publicly  announce  its  contents  earlier,  we  plan  no  further 
distribution  of  this  report  until  30  days  from  the  date  of  this  letter.  At  that 
time,  we  will  provide  copies  to  the  Secretary  of  Transportation;  the  Acting 
Administrator,  faa;  the  Director,  Office  of  Management  and  Budget;  and 
other  interested  parties.  We  will  send  copies  to  others  upon  request. 

Our  work  was  performed  under  directors  in  two  gao  divisions.  Kenneth  M. 
Mead,  Director  of  Transportation  Issues  in  the  Resources,  Community,  and 
Economic  Development  Division,  can  be  reached  at  (202)  275-1000. 

JayEtta  Z.  Hecker,  Director  for  Resources,  Community,  and  Economic 
Development  Information  Systems  in  the  Information  Management  and 
Technology  Division,  can  be  reached  at  (202)  336-6416.  Other  major 
contributors  to  this  report  are  listed  in  appendix  III. 


Sincerely  yours, 


Assistant  Comptroller  General 
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Objectives,  Scope,  and  Methodology 


The  Chairman,  Subcommittee  on  Investigations  and  Oversight,  Committee 
on  Science,  Space,  and  Technology,  asked  us  to  review  a  number  of  issues 
related  to  the  Traffic  Alert/Collision  Avoidance  System  (tcas).  Specifically, 
we  were  asked  to  develop  information  on  (1)  pilots’  and  air  traffic 
controllers’  views  on  tcas,  (2)  faa’s  actions  to  address  tcas’s  problems, 
and  (3)  key  aspects  of  faa’s  software  engineering  approach  for  tcas, 
including  faa’s  plans  to  verify  and  validate  the  system. 

To  obtain  information  on  the  status  of  tcas’s  installation  and  reported 
operational  problems,  we  reviewed  relevant  sections  of  the  Federal 
Aviation  Act  of  1958,  as  amended,  and  faa’s  tcas  regulations,  as  well  as 
advisory  circulars  and  various  reports  and  position  papers  that  provided 
information  on  tcas  operations  and  issues.  We  reviewed  statistics 
developed  by  faa’s  tcas  Transition  Program  to  determine  the  number  of 
altitude  deviations  and  inappropriate  tcas  advisories  reported  by  pilots 
and  air  traffic  controllers. 

We  discussed  tcas’s  operational  problems  with  faa’s  tcas  and  air  traffic 
officials  and  attended  biweekly  meetings  that  faa  instituted  to  keep 
senior-level  management  apprised  of  developments  in  the  tcas  program. 
We  also  attended  meetings  of  various  working  groups,  task  forces,  and 
technical  committees  involved  with  implementing  tcas  to  obtain 
perspectives  from  many  segments  of  the  aviation  community,  including 
pilots,  air  traffic  controllers,  tcas  manufacturers,  airline  officials,  technical 
consultants,  and  aircraft  manufacturers.  To  discuss  reported  problems,  we 
met  with  representatives  of  the  National  Air  Traffic  Controllers 
Association,  Airline  Pilots  Association  and  airlines,  and  with  contractors 
working  on  various  aspects  of  tcas.  We  also  visited  officials  at  Boeing 
Commercial  Airplane  Group  to  obtain  that  company’s  perspective  on  tcas, 
and  we  discussed  tcas’s  reported  operational  problems  with 
representatives  of  each  of  the  three  tcas  manufacturers.  We  also 
interviewed  38  air  traffic  controllers  and  70  pilots.  Because  we 
judgmentally  selected  these  individuals,  our  findings  cannot  be  generalized 
to  the  universe  of  pilots  and  air  traffic  controllers.  In  addition,  to  observe 
tcas  in  operation,  we  rode  in  the  cockpit  “jump  seat”  on  10  Simmons 
Airlines,  Inc./American  Eagle  commuter  flights  into  and  out  of  O’Hare 
Airport,  as  well  as  4  USAir  Express  flights  into  and  out  of  Washington 
National  Airport. 

To  obtain  information  on  tcas  software  development  and  testing,  we  met 
with  faa  and  contractor  officials  involved  in  the  development  and  testing 
of  tcas.  We  also  talked  to  avionics  manufacturers  currently  producing  tcas 
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units.  We  reviewed  tcas  requirements  documentation,  test  plans,  safety 
studies,  test  reports,  certification  standards,  and  verification  and 
validation  plans.  We  interviewed  (1)  faa  officials  at  the  tcas  program 
office  and  aircraft  certification  offices  in  Washington,  D.C.,  and  at  the  faa 
Technical  Center  in  Atlantic  City,  New  Jersey,  and  (2)  software 
development  experts  at  the  Mitre  Corporation  in  McLean,  Virginia,  and  at 
the  Massachusetts  Institute  of  Technology’s  Lincoln  Laboratory  in 
Cambridge,  Massachusetts. 
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Bibliography  of  Software  Engineering 
Standards 


Software  Verification  and  Validation:  Its  Role  in  Computer  Assurance  and 
Its  Relationship  With  Software  Project  Management  Standards.  National 
Institute  of  Standards  and  Technology,  Special  Publication  500-165. 
September  1989. 

Guideline  for  Software  Verification  and  Validation  Plans.  U.S.  Department 
of  Commerce,  National  Bureau  of  Standards,  Federal  Information 
Processing  Standard  132.  November  1987. 

Software  Considerations  in  Airborne  Systems  and  Equipment 
Certification.  Radio  Technical  Commission  for  Aeronautics, 

RTCA/DO- 1 78A.  March  1985. 

Guideline  for  Lifecycle  Validation,  Verification,  and  Testing  of  Computer 
Software.  U.S.  Department  of  Commerce,  National  Bureau  of  Standards, 
Federal  Information  Processing  Standard  101.  June  1983. 

Software  Considerations  in  Airborne  Systems  and  Equipment 
Certification.  Radio  Technical  Commission  for  Aeronautics,  RTCA/DO-178. 
November  1981. 
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